Skip to content

Conversation

@zmedico
Copy link
Contributor

@zmedico zmedico commented Oct 27, 2025

Do not set py_limited_api when Py_GIL_DISABLED == 1, since it will be rejected as shown:

  File "/usr/lib/python3.14t/site-packages/setuptools/_distutils/cmd.py", line 135, in ensure_finalized
    self.finalize_options()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "<string>", line 82, in finalize_options
  File "/usr/lib/python3.14t/site-packages/setuptools/command/bdist_wheel.py", line 250, in finalize_options
    self._validate_py_limited_api()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14t/site-packages/setuptools/command/bdist_wheel.py", line 285, in _validate_py_limited_api
    raise ValueError(
    ...<4 lines>...
    )
ValueError: `py_limited_api='cp314'` not supported. `Py_LIMITED_API` is currently incompatible with `Py_GIL_DISABLED`.See https://github.com/python/cpython/issues/111506.

See python/cpython#111506.

@ngoldbaum ngoldbaum mentioned this pull request Oct 27, 2025
@cclauss
Copy link
Contributor

cclauss commented Oct 30, 2025

@sethmlarson
Copy link
Member

@zmedico @cclauss So this means that Python 3.14t wheels will need to be built from source if we want to publish wheels that support more than one Python version?

@zmedico
Copy link
Contributor Author

zmedico commented Nov 18, 2025

@zmedico @cclauss So this means that Python 3.14t wheels will need to be built from source if we want to publish wheels that support more than one Python version?

Correct, and this will be the case until a new stable ABI is available to target. A PEP is needed in order to create the new stable ABI, and there's some initial discussion about it here:

https://discuss.python.org/t/making-pyobject-opaque-in-the-limited-api/77206

@cclauss
Copy link
Contributor

cclauss commented Nov 18, 2025

Please rebase now the #205 has been merged.

@zmedico zmedico force-pushed the support-Py_GIL_DISABLED branch from b750891 to 41a90cd Compare November 18, 2025 06:08
@cclauss
Copy link
Contributor

cclauss commented Nov 18, 2025

@sethmlarson Can you please approve the GitHub Actions workflow below so we can see if the tests pass?

@ngoldbaum
Copy link
Contributor

Hi all, I was planning to work on this, along with updating all the release automation to support building cp314t wheels (if that's ok with @sethmlarson).

IMO it's probably not a good idea to merge this without at least adding a cp314t testing job.

@zmedico zmedico force-pushed the support-Py_GIL_DISABLED branch 2 times, most recently from fdaa76b to fc573cc Compare November 18, 2025 20:07
@cclauss
Copy link
Contributor

cclauss commented Nov 18, 2025

IMO it's probably not a good idea to merge this without at least adding a cp314t testing job.

The double negatives make that line a bit confusing. I believe that you intend to say:

IMO it's a good idea to add a cp314t testing job before merging this.

Should rebase to pick up the Py3.13 and 3.14 tests landed in:

Oh... https://www.githubstatus.com

@ngoldbaum
Copy link
Contributor

IMO it's a good idea to add a cp314t testing job before merging this.

Yup, that's what I meant.

@zmedico zmedico force-pushed the support-Py_GIL_DISABLED branch from fc573cc to ebd1aa9 Compare November 18, 2025 21:43
@cclauss cclauss requested a review from ngoldbaum November 18, 2025 22:19
@cclauss
Copy link
Contributor

cclauss commented Nov 18, 2025

Please add the change in #210 to this pull request.

Do not set py_limited_api when Py_GIL_DISABLED == 1, since it
will be rejected as shown:

  File "/usr/lib/python3.14t/site-packages/setuptools/_distutils/cmd.py", line 135, in ensure_finalized
    self.finalize_options()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "<string>", line 82, in finalize_options
  File "/usr/lib/python3.14t/site-packages/setuptools/command/bdist_wheel.py", line 250, in finalize_options
    self._validate_py_limited_api()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.14t/site-packages/setuptools/command/bdist_wheel.py", line 285, in _validate_py_limited_api
    raise ValueError(
    ...<4 lines>...
    )
ValueError: `py_limited_api='cp314'` not supported. `Py_LIMITED_API` is currently incompatible with `Py_GIL_DISABLED`.See python/cpython#111506.
@zmedico zmedico force-pushed the support-Py_GIL_DISABLED branch from ebd1aa9 to 2dd9a6c Compare November 18, 2025 23:40
@cclauss cclauss requested a review from sethmlarson November 19, 2025 00:30
@sethmlarson
Copy link
Member

@ngoldbaum Is this the correct way to handle this situation for free-threading? Wouldn't another option be to disable the limited API altogether if free-threading was enabled? You would know better than I, probably.

@cclauss
Copy link
Contributor

cclauss commented Nov 19, 2025

We want to be testing free-threading (this PR) before declaring the code to be compatible.

My bet is that we will want to be running pytest-run-parallel after this PR lands.

@sethmlarson
Copy link
Member

@cclauss @ngoldbaum Is there value in getting a 1.2.0.0 release out ahead of adding support for free-threading? We can release a 1.2.0.1 that adds support for free-threading at any point.

@ngoldbaum
Copy link
Contributor

Hi, sorry I haven't looked at anything over here yet.

I'm hoping to carve some time out today but otherwise I should have time later this week.

Please don't feel like cp314t support is necessary to do a release. As you said, that can come in a bugfix release.

@cclauss cclauss merged commit ac7cb9e into python-hyper:main Nov 19, 2025
30 checks passed
@sethmlarson
Copy link
Member

@cclauss Is merging this PR confirming that this is the right way to handle this? See: #206 (comment) I was expecting maybe a reference or a link to another project in this same scenario as us.

@cclauss
Copy link
Contributor

cclauss commented Nov 19, 2025

Current state: We are running tests against python3.14t on every code change. We are not saying that the code is compatible with free-threading. We are merely saying that our current tests pass on free-threaded Python.

We can now add more tests to help us be more confident in free-threading compatibility. As we progress, automated tests will tell us if things break with and without the GIL. nltk/nltk is a project that is in a similar state.

@ngoldbaum
Copy link
Contributor

@cclauss in the future I'd appreciate it if you could wait for me to review code. Particularly when I indicate that I want to look at stuff but don't have time.

@sethmlarson
Copy link
Member

Yeah my initial thought now that this is merged is that we should maybe... revert it until we get a review from @ngoldbaum? I want to make absolutely sure our build config is correct, because anything we do others are liable to copy, too, if this is not well-trodden ground.

@cclauss
Copy link
Contributor

cclauss commented Nov 19, 2025

I thought @zmedico's description in the commit message was convincing. Given that this change only affects free-threaded builds (which we do not yet support) I do not see how this change has any effect.

@sethmlarson
Copy link
Member

So saying we don't support something doesn't mean that people won't use or try it :) I would actually prefer things keep on breaking until we have a solution we're happy with and are convinced its correct.

@ngoldbaum
Copy link
Contributor

I would also prefer backing it out.

You could take a look at the PR where we enable cp314t wheel builds in pynacl, as an example: pyca/pynacl#880

It looks like over there the py_limited_api setting is passed in via pip. I'm not sure what setting py_limited_api to cp314t does. It may work. I'd rather just not set it on the free-threaded build, I think that's clearer. The default in cffi is to build version-specific wheels.

I'd like to do a pass over the wheel-building configuration in this repository to hopefully make it as painless as possible to ship version-specific wheels on the free-threaded build.

@cclauss please please give me some time to look at things and work on things. I will be looking at shipping cp314t wheels, it's just not an emergency and it's ok if it takes a little while to get done. Now that you and I can trigger builds it will be much quicker. You can also rely on my experience here instead of trying to figure out everything on your own.

I also think @sethmlarson would prefer if we hold off on the free-threaded stuff until after v1.2.0 ships.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants